Combine harvester control information for a remote user with visual feed

ABSTRACT

A user device, that is remote from a combine harvester, communicates with the remote harvester to receive contextual information indicative of machine settings on the combine harvester. Remote view and control logic receives the contextual information from the combine harvester, along with image or video display information generated from an image capture device (such as a video camera or other image capture device) on the combine harvester. The contextual information is displayed, along with the video or image information on the remote user device.

FIELD OF THE DESCRIPTION

The present description relates to a control interface for anagricultural machine. More specifically, the present description relatesto a control interface for a remote operator of a combine harvester,that includes image or video data.

BACKGROUND

There are a wide variety of different types of equipment, such asconstruction equipment, turf management equipment, forestry equipment,and agricultural equipment. These types of equipment are operated by anoperator. For instance, a combine harvester (or combine) is operated byan operator, and it has many different mechanisms that are controlled bythe operator in performing a harvesting operation. The combine may havemultiple different mechanical, electrical, hydraulic, pneumatic,electro-mechanical (or other) subsystems, some or all of which can becontrolled, at least to some extent, by the operator.

The systems may need the operator to make a manual adjustment outsidethe operator's compartment or to set a wide variety of differentsettings and provide various control inputs in order to control thecombine. Some inputs not only include controlling the combine directionand speed, but also threshing clearance and sieve and chaffer settings,rotor and fan speed settings, and a wide variety of other settings andcontrol inputs.

Because of the complex nature of the combine operation, it can be verydifficult to know how a particular operator or machine is performing ina given harvesting operation. While some systems are currently availablethat sense some operational and other characteristics, and make themavailable to reviewing personnel, those systems are normallyinformational in nature.

The discussion above is merely provided for general backgroundinformation and is not intended to be used as an aid in determining thescope of the claimed subject matter.

SUMMARY

A user device, that is remote from a combine harvester, communicateswith the remote harvester to receive contextual information indicativeof machine settings on the combine harvester. Remote view and controllogic receives the contextual information from the combine harvester,along with image or video display information generated from an imagecapture device (such as a video camera or other image capture device) onthe combine harvester. The contextual information is displayed, alongwith the video or image information on the remote user device.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter. The claimed subject matter is not limited to implementationsthat solve any or all disadvantages noted in the background.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a partial pictorial, partial schematic illustration of acombine harvester.

FIG. 2 is a block diagram of one example of a computing systemarchitecture that includes the combine harvester illustrated in FIG. 1.

FIG. 3 is a block diagram showing one example of a context andimage/video transmission system, in more detail.

FIG. 3A is a flow diagram illustrating one example of the operation ofthe transmission system shown in FIG. 3.

FIG. 4 is a block diagram showing one example of remote view/controllogic, in more detail.

FIG. 4A is a flow diagram illustrating one example of the operation ofthe logic shown in FIG. 4.

FIG. 5 is a block diagram showing one example of a mobile device (remoteuser computing system) with a control and monitor user interface displaythat can be generated on the mobile device.

FIG. 6 shows one example of a multiple machine display that can begenerated on the remote user computing system.

FIGS. 7 and 8 show additional examples of control and monitor userinterface displays that can be generated on the remote user computingsystem.

FIG. 9 shows one example of the architecture illustrated in FIG. 2,deployed in a remote server architecture.

FIGS. 10-12 show examples of mobile devices that can be used in thearchitectures shown in the previous figures.

FIG. 13 is a block diagram showing one example of a computingenvironment that can be used in the architectures shown in the previousfigures.

DETAILED DESCRIPTION

Combine harvesters often have a wide variety of sensors or inputs thatsense or identify a variety of different variables, such as operatingparameters, machine settings, machine configuration, cropcharacteristics, environmental parameters, etc. The sensors cancommunicate this information over a controller area network (CAN) bus(or another network, such as an Ethernet network, or even wirelessnetworks, etc.) to various systems that can process the sensor signalsand generate output signals (such as control signals) based on thesensed variables. Given the complex nature of the control operationsneeded to operate a combine harvester, and given the wide variety ofdifferent types of settings and adjustments that an operator can make,and further given the widely varying different types of crops, terrain,crop characteristics, etc. that can be encountered by a combineharvester, it can be very difficult to determine how a particularmachine, or operator, is performing and why. This problem is exacerbatedwhen a particular organization has a plurality of different combineharvesters that are all operating at the same time. These combineharvesters are often referred to as a “fleet” of harvesters.

The operation of the fleet of harvesters is often overseen by a (remoteor local) fleet manager (or farm manager) who is located remotelyrelative to at least some of the combine harvesters in the fleet. It canbe extremely difficult for a farm manager or remote manager to determinehow the various combine harvesters are operating in the fleet, how theyare operating relative to one another, how they are operating relativeto other similarly situated harvesters, etc.

It is also extremely difficult for a remote manager to identifyperformance criteria for the various operators and machines, anddetermine how they compare relative to one another, in near real time.Thus, it is very difficult for a remote manager to attempt to modify thesettings on any combine harvester to increase the performance of thatharvester. This is because the remote manager does not have access tothe current settings of a particular machine, nor does the remotemanager have access to an interface that allows the remote manager toview and interact with display elements that indicate how variousmachines and operators are performing relative to one another.

Instead, the remote manager often needs to review data after theharvesting season, and even then the task is difficult. The remotemanager often needs to switch between different applications, betweendifferent views of data, for the different machines and operators, in anattempt to compare the data in this way. This results in a relativelylarge amount of bandwidth consumption, because the operator often needsto make many different calls from his or her device to a remote datastore where the information is stored.

Some systems currently allow remote viewing of settings, to some extent.One drawback is the delay time involved. In current systems, there maybe a delay of thirty minutes or more. Even if the machine settings wereshown in real time (or near real time), they are often represented asnumeric values which can be difficult to interpret. The presentdescription thus describes a remote user interface that shows bothcurrent data (such as machine settings or sensor data) along withrelevant image or video data. A user actuatable control input mechanismcan also be provided for remote control of the agricultural machine.

FIG. 1 is a partial pictorial, partial schematic, illustration of anagricultural machine 100, in an example where machine 100 is a combineharvester (or combine). It can be seen in FIG. 1 that combine 100illustratively includes an operator compartment 101, which can have avariety of different operator interface mechanisms, for controllingcombine 100, as will be discussed in more detail below. Combine 100 caninclude a set of front end equipment that can include header 102, and acutter generally indicated at 104. It can also include a feeder house106, a feed accelerator 108, and a thresher generally indicated at 110.Thresher 110 illustratively includes a threshing rotor 112 and a set ofconcaves 114. Further, combine 100 can include a separator 116 thatincludes a separator rotor. Combine 100 can include a cleaning subsystem(or cleaning shoe) 118 that, itself, can include a cleaning fan 120,chaffer 122 and sieve 124. The material handling subsystem in combine100 can include (in addition to a feeder house 106 and feed accelerator108) discharge beater 126, tailings elevator 128, clean grain elevator130 (that moves clean grain into clean grain tank 132) as well asunloading auger 134 and spout 136. Combine 100 can further include aresidue subsystem 138 that can include chopper 140 and spreader 142.Combine 100 can also have a propulsion subsystem that includes an enginethat drives ground engaging wheels 144 or tracks, etc. It will be notedthat combine 100 may also have more than one of any of the subsystemsmentioned above (such as left and right cleaning shoes, separators,etc.).

In operation, and by way of overview, combine 100 illustratively movesthrough a field in the direction indicated by arrow 146. As it moves,header 102 engages the crop to be harvested and gathers it toward cutter104. After it is cut, it is moved through a conveyor in feeder house 106toward feed accelerator 108, which accelerates the crop into thresher110. The crop is threshed by rotor 112 rotating the crop against concave114. The threshed crop is moved by a separator rotor in separator 116where some of the residue is moved by discharge beater 126 toward theresidue subsystem 138. It can be chopped by residue chopper 140 andspread on the field by spreader 142. In other implementations, theresidue is simply dropped in a windrow, instead of being chopped andspread.

Grain falls to cleaning shoe (or cleaning subsystem) 118. Chaffer 122separates some of the larger material from the grain, and sieve 124separates some of the finer material from the clean grain. Clean grainfalls to an auger in clean grain elevator 130, which moves the cleangrain upward and deposits it in clean grain tank 132. Residue can beremoved from the cleaning shoe 118 by airflow generated by cleaning fan120. That residue can also be moved rearwardly in combine 100 toward theresidue handling subsystem 138.

Tailings can be moved by tailings elevator 128 back to thresher 110where they can be re-threshed. Alternatively, the tailings can also bepassed to a separate re-threshing mechanism (also using a tailingselevator or another transport mechanism) where they can be re-threshedas well.

FIG. 1 also shows that, in one example, combine 100 can include groundspeed sensor 147, one or more separator loss sensors 148, a clean graincamera 150, one or more cleaning shoe loss sensors 152, forward lookingcamera 154, rearward looking camera 156, a tailings elevator camera 158,and a wide variety of other cameras or image/video capture devices.Ground speed sensor 146 illustratively senses the travel speed ofcombine 100 over the ground. This can be done by sensing the speed ofrotation of the wheels, the drive shaft, the axel, or other components.The travel speed can also be sensed by a positioning system, such as aglobal positioning system (GPS), a dead reckoning system, a LORANsystem, or a wide variety of other systems or sensors that provide anindication of travel speed.

Cleaning shoe loss sensors 152 illustratively provide an output signalindicative of the quantity of grain loss by both the right and leftsides of the cleaning shoe 118. In one example, sensors 152 are strikesensors which count grain strikes per unit of time (or per unit ofdistance traveled) to provide an indication of the cleaning shoe grainloss. The strike sensors for the right and left sides of the cleaningshoe can provide individual signals, or a combined or aggregated signal.It will be noted that sensors 152 can comprise only a single sensor aswell, instead of separate sensors for each shoe.

Separator loss sensor 148 provides a signal indicative of grain loss inthe left and right separators. The sensors associated with the left andright separators can provide separate grain loss signals or a combinedor aggregate signal. This can be done using a wide variety of differenttypes of sensors as well. It will be noted that separator loss sensors148 may also comprise only a single sensor, instead of separate left andright sensors.

Cameras 150, 154, 156 and 158 illustratively capture video or stillimages that can be transmitted to, and displayed on, a display inoperator compartment 101 or a remote device (shown in more detail below)in near real time. Clean grain camera 150, for instance, generates avideo feed showing grain passing into clean grain tank 132 (or throughclean grain elevator 130). Camera 154 can illustratively generate avideo feed showing a view forward of operator compartment 101, such asshowing header 102 and/or the crop in front of header 102. Cameras 156and 158 illustratively generate a video feed showing the tailings inelevator 158 and the discharge beater 142 and an area of the fieldbehind combine 100, respectively. These are examples only, andadditional or different cameras can be used and/or they can be devicesthat capture still images or other visual data.

It will also be appreciated that sensor and measurement mechanisms (inaddition to the sensors, cameras, etc. already described) can includeother sensors on combine 100 as well. For instance, they can include aresidue setting sensor that is configured to sense whether machine 100is configured to chop the residue, drop a windrow, etc. They can includecleaning shoe fan speed sensors that can be configured proximate fan 120to sense the speed of the fan. They can include a threshing clearancesensor that senses clearance between the rotor 112 and concaves 114.They include a threshing rotor speed sensor that senses a rotor speed ofrotor 112. They can include a chaffer clearance sensor that senses thesize of openings in chaffer 122. They can include a sieve clearancesensor that senses the size of openings in sieve 124. They can include amaterial other than grain (MOG) moisture sensor that can be configuredto sense the moisture level of the material other than grain that ispassing through combine 100. They can include machine setting sensorsthat are configured to sense the various configurable settings oncombine 100. They can also include a machine orientation sensor that canbe any of a wide variety of different types of sensors that sense theorientation of combine 100. Crop property sensors can sense a variety ofdifferent types of crop properties, such as crop type, crop moisture,and other crop properties. They can also be configured to sensecharacteristics of the crop as it is being processed by combine 100. Forinstance, they can sense grain feed rate, as it travels through cleangrain elevator 130. They can sense mass flow rate of grain throughelevator 130, or provide other output signals indicative of other sensedvariables. Some additional examples of the types of sensors that can beused are described below.

FIG. 2 is a block diagram showing one example of an architecture 200that includes combine harvester 100 coupled for communication withremote server computing system 202 and remote user computing 204, overnetwork 206. Network 206 can be any of a wide variety of different typesof networks, such as a wide area network, a local area network, a nearfield communication network, a cellular network, or any of a widevariety of other networks or combinations of networks. As is discussedin greater detail below, combine harvester 100 can communicate withother systems using store-and-forward mechanisms as well. FIG. 2 alsoshows that, in one example, combine harvester 100 can generate operatorinterface displays 208 with user input mechanisms 210 for interaction byoperator 212. Operator 212 is illustratively a local operator of combine100, in the operator's compartment 101 of combine 100, and can interactwith user input mechanisms 210 in order to control and manipulatecombine harvester 100. In addition, as is described below, operator 212can interact directly with other user interface mechanisms on combineharvester 100. This is indicated by arrow 214.

FIG. 2 also shows that, in one example, remote user computing system 204illustratively generates user interfaces 216, with user input mechanisms218, for interaction by remote user 220 (who may be a farm manager, aremote manager, or other remote user that has access to datacorresponding to combine 100). Remote user 220 illustratively interactswith user input mechanisms 218 in order to control and manipulate remoteuser computing system 204, and, in some examples, to control portions ofcombine harvester 100 and/or remote server computing system 202.

In the example shown in FIG. 2, agricultural harvesting machine (e.g.,combine) 100 includes one or more processors or servers 222, sensors224, data store 226, user interface mechanisms 228, context andimage/video transmission system (transmission system) 230, remotecontrol signal processing logic 232, communication system 234, controlsystem 236, controllable subsystems 238, and it can include a widevariety of other items 240. Communication system 234 can includeon-board communication logic 242 (which, itself, can include such thingsas a controller area network (CAN) bus, a Wi-Fi network, an Ethernetnetwork, or any of a wide variety of other wired or wirelesscommunication systems or networks or combinations of networks), that isused to facilitate communication of items on board machine 100.Communication system 234 can also include remote communication logic 244(which, itself, can include a cellular communication network, asatellite communication system, a store-and-forward communicationsystem, a large area network communication system, or a wide variety ofother wired or wireless communication systems or a combination ofsystems), that is used to facilitate communication with items remotefrom machine 100. Communication system 234 can include other items 246.Sensors 224 can include any of the sensors discussed above with respectto FIG. 1. By way of example, they can include image/video capturesensors 226 (which, themselves, can be any of the cameras 150, 154, 156,158, or other video or image capture sensors), and they can include awide variety of other sensors 228 (which may be any or all of thesensors discussed above and/or other sensors)).

User interface mechanisms 228 can include one or more display devices,audio devices, one or more haptic devices, and it can include otheritems, such as a steering wheel, one or more joysticks, pedals, levers,buttons, keypads, etc. Where the user interface mechanisms include auser interface display 208, then user input mechanisms 210 can includebuttons, icons, actuatable links, or other items that can be actuated byoperator 212. When the control system 236 or other items on machine 100use speech recognition, and/or speech synthesis, then user interfacemechanisms 228 can include a microphone, a speaker, etc.

Control system 236 can include logic and actuators or other items thatcan perform various types of processing and generate control signals tocontrol controllable subsystems 238. The control signals can begenerated based on user inputs, they can be generated automaticallybased on sensor inputs, based on detected events or otherwise. They canalso be generated based on remote control inputs received from remoteuser computing system 204 and from remote control signal processinglogic 232. Controllable subsystems 238 can illustratively include suchthings as rotor 112, concaves 114, cleaning fan 120, chaffer 122, sieve124, a propulsion system 250, and a steering system 252. It can includea wide variety of other items 254 as well.

Context and image video transmission system 230 receives a request forcontext and video or image data, and obtains that information fromimage/video capture mechanisms 226 and other sensors 228, or othersources of context information. It generates a transmission message andtransmits it to remote user system 204 through network 206. This isdescribed in greater detail below.

Remote control signal processing logic 232 receives remote controlsignals over network 206, from remote user system 204. It processesthose signals and provides them to control system 236. Control system236 can then control the controllable subsystems 238 (or other items)based upon the signals received from remote control signal processinglogic 232.

In the example shown in FIG. 2, remote server computing system 202illustratively includes one or more processors or servers 260, datastore 262, authentication system 264, communication logic 266, and itcan include a wide variety of other items 268. Authentication system 264can be used to authenticate different operators 212 or remote users 220,so that they can access resources available through remote servercomputing system 202. Communication logic 266 illustratively allowscommunication, over network 206, with both remote user system 204 andcombine 100. It illustratively includes communication logic that enablescommunication of video or image information. It also illustrativelyincludes context communication logic which enables communication of thecontext information discussed above.

Remote user computing system 204 can be a wide variety of differenttypes of systems, such as a mobile device, a laptop computer, a desktopcomputer, etc. It illustratively includes one or more processors orservers 270, data store 272, communication system 274, remoteview/control logic 276, user interface logic 278, and it can include awide variety of other items 280.

Communication system 274 illustratively allows remote user computingsystem 204 to communicate with remote server computing system 202 andwith combine 100 over network 206. Therefore, it can include one or moredifferent communication systems, such as a cellular communicationsystem, a satellite communication system, a wide area networkcommunication system, a local area network communication system, a nearfield communication system, or a variety of other communication systemsor combinations of systems.

Remote view/control logic 276 illustratively generates user interfaces216 that include both the context information and the correspondingvideo or image data. It also illustratively generates a user interfacewith one or more user control inputs that remote user 220 can actuate inorder to adjust the settings on combine 100 or to otherwise controlcombine 100. User interface logic 278 illustratively surfaces (ordisplays) the displays generated by logic 276 so that remote user 220can view and control combine 100.

FIG. 3 is a block diagram showing one example of transmission system 230(on combine 100 in FIG. 2) in more detail. System 230 illustrativelyincludes trigger detection logic 282, video/image processing logic 284,image or video/context information correlation logic 286, outputgenerator logic 288, and it can include other items 290. Triggerdetection logic 282, itself, can include continuous transmissiondetector 292, and periodic/intermittent transmission detector 294.Trigger detection logic 282 detects one or more different triggers thatindicate that system 230 should obtain and send image or video datacaptured by image/video capture mechanisms 226, along with thecorresponding context information, and generates a trigger signal toperform that process. Continuous transmission detector 290 detects aninput or other indicator that indicates that the video/image and contextdata should be sent continuously in near real time, and generates thetrigger signal indicating this. Periodic/intermittent transmissiondetector 294 detects an input indicating that the video or image andcontext data should be sent periodically, or intermittently, andgenerates the corresponding trigger signal. Event detector 296 detectsevents, and generates the corresponding trigger signal, which triggersthe transmission of the information. For instance, it may be that system230 is configured so that, every time an operator makes a settingschange, this is detected by event detector 296. In response to thecorresponding trigger signal, the context information indicative of thatsettings change, and the corresponding image or video information, istransmitted to remote user computing system 204. It can be sentcontinuously, in near real time, until another trigger is detected, orit can be sent in other ways. When events are detected indicating asetting change, event detector 296 can be configured to detect whensettings are changed by the operator. Event detector 296 can beconfigured to detect a wide variety of other events as well.

Request detector 298 can be configured to detect when the video or imageand corresponding context information is to be transmitted, on demand,based on a request from remote user 220, through remote user computingsystem 204. For instance, it may be that remote user 220 wishes to viewthe image or video and corresponding context information from one ormore combines 100. In that case, remote view/control logic 276 (shown inFIG. 2) can surface a user interface for remote user 220 so that remoteuser 220 can select one or more different combines 100, and generate arequest to view the image or video and corresponding context informationfor the selected combines. That request can be sent through network 206(and possibly through remote server computing system 202) totransmission system 230, where request detector 298 detects it as atransmission trigger and generates the trigger signal.

In response to a transmission trigger being detected by logic 282,video/image processing logic 284 obtains video or image data from videoor image capture mechanisms 226 (e.g., cameras 150, 154, 156, 158, etc.)and generates a message (or video transmission) that can be transmittedto remote user computing system 204 over network 206. In addition, imageor video/context information correlation logic 286 identifies theparticular context information that corresponds to the image or videoinformation that is being processed and transmitted to remote usercomputing system 204.

For instance, it may be that a grain quality sensor has sensed that arelatively large amount of grain that is entering the clean grain tankof combine 100 is cracked or otherwise damaged. This can be done, forinstance, by performing image processing on images taken by the camera150 in the clean grain elevator. A grain quality metric can be generatedthat is indicative of the quality of the grain entering the clean graintank, in terms of the amount of grain that is damaged. Therefore, it maybe that remote user 220 has seen an decrease in the grain qualitymetric. In that case, remote user 220 may generate a request to see thesensor data corresponding to the grain quality metric, along with thevideo from camera 150 showing actual grain entering the clean graintank. In this way, user 220 sees not only a numeric or graphicalindicator that indicates the quality of the grain entering the cleangrain tank, but also sees a video image showing that grain, in near realtime.

Remote user 220 can then provide a control input to make adjustments toone of the controllable subsystems 238 in order to improve grainquality, or remote user 220 can communicate with operator 212 requestingthat operator 212 make such an adjustment. Remote user 220 can thencontinue to monitor the video information from camera 150, along withthe corresponding context information (e.g., the grain quality metricand the outputs of any other sensors that may correspond to, or relateto, the grain quality being harvested). Therefore, once an indicationhas been received that indicates the particular video or imageinformation that is to be transmitted, correlation logic 286 thenidentifies the corresponding context information that is to be sentalong with that image or video information. Output generator logic 288illustratively generates an output containing the image or videoinformation as well as the corresponding context information. Thatoutput can be sent to remote user computing system 204 through network206, using communication system 234.

FIG. 3A is a flow diagram illustrating one example of the operation oftransmission system 230 in FIG. 3. It is first assumed that transmissionsystem 230 is configured to detect a trigger to send video or image datain conjunction with its corresponding context data, to a remote usercomputing system 204. Therefore, trigger detection logic 282 firstdetects a trigger to send image and/or video data. This is indicated byblock 350 in the flow diagram of FIG. 3A. Again, this can be any numberof different triggers. Continuous transmission detector 292 may detectan input indicating that the image or video data should be sentcontinuously until otherwise triggered. This is indicated by block 352.Periodic/intermittent transmission detector 294 can detect an inputindicating that it is time to send the video or image data. Forinstance, it may be that a timer is set and periodically, whenever thetimer goes off, the video or image data is to be transmitted. In anotherexample, it may be intermittent, but not periodic. Periodic/intermittenttrigger detection is indicated by block 354.

Event detector 296 may detect an event that triggers the transmission ofimage or video data. For instance, if the operator of combine 100suddenly changes speeds, if the yield of harvested crop suddenly dropsor changes by an amount above a threshold amount, or if the operator hasmade a settings change, or any of a wide variety of other events occur,then event trigger detector 296 may detect a trigger indicating thatvideo or image data is to be transmitted. This is indicated by block356.

Request detector 298 may detect a request for video or image data from aremote user computing system 204. This is indicated by block 358. A widevariety of other triggers can be detected in other ways as well. This isindicated by block 360. Once the trigger has been detected, thenvideo/image processing logic 284 identifies the particular video orimage data that is to be sent, and correlation logic 286 identifies thecorresponding context information that is to be sent along with thevideo or image data. Identifying the video or image data to be sent andthe corresponding context information is indicated by block 362. Thiscan be done by accessing a set of mappings that map context data tovideo or image data. This is indicated by block 364. The video or imagedata can be specifically requested in the request from a remote usercomputing system 204, and the context information can be specificallyrequested as well. Identifying the video or image data and thecorresponding context information based on the request is indicated byblock 366. The information can be identified in other ways as well. Thisis indicated by block 368.

Output generator logic 288 then generates an output indicative of theinformation to be sent. This is indicated by block 370. For instance, itcan obtain the video or image data (or a live feed) from the camerasmentioned above. This is indicated by block 372. It can obtain contextinformation from various sensors or other identifying mechanisms. Thisis indicated by block 374. It can generate information to send in otherways as well, and this is indicated by block 376.

Output generator logic 288 then begins sending the information throughcommunication system 234 and network 206 to remote user computing system204, as desired or indicated. This is indicated by block 278. Forinstance, it can send continuously streaming data as indicated by block380. It can send an initial set of data, and then data updatesintermittently, such as in bursts, periodically, or in other ways. Thisis indicated by block 382. It can send updated information based onother triggered events or requests received. This is indicated by block384. It can send the video or image data and corresponding contextinformation in other ways as well, and this is indicated by block 386.

Remote control signal processing logic 232 also illustratively receivesand processes any control inputs. This is indicated by block 388. Forinstance, if remote user 220 sends a control input to change thesettings on combine 100, or to remotely control it in other ways, thisis received and processed by logic 232.

FIG. 4 is a block diagram showing one example of remote view/controllogic 276 (shown on remote user computing system 204 in FIG. 2). In theexample shown in FIG. 4, logic 276 illustratively includes machineselector logic 302, contextual information display generator logic 304,image/video display generator logic 306, remote control input mechanismgenerator logic 308, user interaction detector 310, and it can include awide variety of other items 312.

Machine selector logic 302 can generate a user interface mechanism thatcan be actuated by remote user 220 in order to select one or morecombines from which image or video and corresponding context informationis to be obtained and displayed. For instance, it can access remoteserver computing system 202 to identify the particular machines thatremote user 220 has access to, after remote user 220 has authenticatedto remote server computing system 202 through authentication system 264.Once the machines that remote user 220 has access to have beenidentified, then machine selector logic 302 can generate a userinterface display, with a user input mechanism that allows remote user220 to select one or more of those machines for which a display is to begenerated. Based upon the user inputs, request generator logic 303 thengenerates a request for image or video and corresponding contextinformation from the selected combines 100. The request can be sentthrough remote server computing system 202, so that the request can beauthenticated and authorized and then sent on to the selected combines100, or it can be sent directly to those combines.

Contextual information display generator logic 302 illustrativelyreceives the response that includes the context information. Itgenerates a display element showing the contextual information thatcorresponds to the image or video information that is to be displayed.Image/video display generator logic 306 illustratively receives theimage or video information and generates a display corresponding to thatinformation. Thus, a user interface display can be provided to remoteuser 220, through a display device on remote user computing system 204,that shows the image or video information, along with the correspondingcontext information.

Remote control input mechanism generator logic 308 illustrativelygenerates a remote control user input mechanism that can also bedisplayed or otherwise surfaced to the user, and actuated by remote user220. User interaction detector 310 illustratively detects userinteraction with the user interface display so that appropriate actionscan be taken. For instance, it may be that remote user 220 makes anotherrequest to see video or image and corresponding context information froma different combine, or to see different image or video andcorresponding context information from the same combine or combines.That interaction is detected by detector 310 so that request generatorlogic 303 can generate the appropriate request. In addition, it may bethat remote user 220 interacts with the control input mechanism on theuser interface display in order to adjust settings on combine 100, or tootherwise remotely control combine 100. In that case, user interactiondetector 310 detects that interaction and provides it to requestgenerator logic 302 which generates a control output that is transmittedto remote control signal processing logic 232 on the combine 100 that isto be remotely controlled or adjusted.

FIG. 4A is a flow diagram illustrating one example of the operation ofremote view/control logic 276, that resides on remote user computingsystem 204, in more detail. It is first assumed that remote operator 220has authenticated himself or herself to remote user computing system 204and/or to remote server computing system 202. At some point, remote user220 will wish to see video or image data, and corresponding contextinformation, from one or more harvesting machines. In that case, inresponse to a user input, machine selector logic 302 illustrativelydisplays a machine and/or data selector that can be actuated by remoteuser 220 to select one or more machines, and to select the type of datato be displayed. This is indicated by block 390 in the flow diagram ofFIG. 4A. In one example, machine selector logic 302 accesses remoteserver computing system 202 to identify which particular machines anddata remote user 220 is authorized to see and control remotely.Displaying a selector for selecting authorized machines and authorizeddata is indicated by block 392. The machine and data selector can bedisplayed in other ways as well, and this is indicated by block 394.

User interaction detector 310 then detects user interaction with themachine/data selector. This interaction illustratively identifies theparticular machine or machines for which data is to be viewed, and theparticular data that is to be obtained from that set of machines, and tobe displayed to remote user 220. Detecting the user interactionselecting machines and data to view is indicated by block 396.

Based on that information, request generator logic 303 generates andsends a request to the selected machine(s), for the identified data.This is indicated by block 398.

Contextual information display generator logic 304 and image/videodisplay generator logic 306 then receive a response to the request andgenerate a display of the image or video information and thecorresponding context information. Receiving the response and generatingthe display is indicated by blocks 400 and 402, respectively, in theflow diagram of FIG. 4A. It will be noted that the data received anddisplayed can be for a single machine, or for multiple differentmachines. This is indicated by block 404. The display can include thecontext information as indicated by block 406 and the video or imagedata as indicated by block 408.

Remote control input mechanism generator logic 308 also illustrativelygenerates a remote control input mechanism that is displayed and thatcan be actuated by remote user 220 to perform the remote settingsadjustment or remote control of machine 100. This is indicated by block410. The display can be generated in other ways, with other displayelements or actuators. This is indicated by block 412.

User interaction detector 310 then detects and processes any userinteractions with the display. This is indicated by block 414. Forinstance, remote user 220 may interact with the control input to performa control operation on one or more of the machines. This is indicated byblock 416. Remote operator 220 may interact with a navigation inputnavigating the user to a different display or to drill down into themore detailed information about the display shown or to navigate inother ways. This is indicated by block 418. The user interaction can bedetected in other ways as well. This is indicated by block 420.

FIG. 5 shows one example of a mobile device 314 that can include remoteuser computing system 204. Mobile device 314 illustratively includes adisplay screen that generates a user interface display 316. Userinterface display 316 illustratively includes a video/image displayportion 318, a contextual information display portion 320, a controlinput mechanism 322, and it can include a wide variety of other items324. The video/image display portion 318 illustratively displays thevideo or image information while contextual information display portion320 displays the corresponding context information. In the example shownin FIG. 5, the contextual information display portion 320 is showngenerating a numerical display 326 and a graphical display 328, althoughit can generate a wide variety of other displays. Control inputmechanism 322 can be a mechanism that is actuated by remote user 220 inorder to directly change a setting or to otherwise control the selectedcombine, or it can be an actuator that navigates the user to anotheruser interface where additional control input mechanisms are provided tocontrol different portions of combine 100.

FIG. 6 shows another example of a user interface that can be generated.In FIG. 6, a computing device 330 can include, for instance, a desktopcomputer, a laptop computer, a tablet computer, etc. It includes adisplay screen that generates a user interface display 332 that showsvideo or image, and corresponding context information, from a pluralityof different combines. For instance, where remote user 220 has providedinputs selecting a plurality of different combines 100, then a userinterface display can be generated corresponding to each of the selectedcombines. In the example shown in FIG. 6, user interface display 332illustratively includes a plurality of different machine displayportions 334, 336, and 338, each of which display video and imageinformation, along with corresponding context information, for adifferent machine. Display portion 334 illustratively includesvideo/image display portion 340 that shows video or image informationfor a first machine, along with a context information display portion342 that displays corresponding context information. Display portion 334also illustratively includes a control input mechanism 344 that can beactuated in order to make a settings adjustment, or to otherwise controlthe first machine. Display portions 336 and 338 illustratively includesimilar display portions, but for information obtained from differentmachines. It will also be noted that display portions 336 and 338 candisplay different video or image information and different contextinformation for the different machines, or it can display informationsimilar to that displayed for the first machine in display portion 334.All of these and other configurations are contemplated herein.

FIGS. 7 and 8 show two other examples of user interface displays thatcan be generated on remote user computing system 204. For instance, FIG.7 shows a user interface display 430 that can be generated on a mobiledevice. Display 430 illustratively includes a video display portion 432that shows live streaming video information generated from the cleangrain camera 150 on combine 100. It also illustratively includes amachine selector 434 that can be used to scroll to different machines,or to different cameras on the same machine. Context information displayportion 438 displays context information corresponding to theinformation generated by the clean grain camera 150. For instance, itincludes a current settings portion 439 that shows values correspondingto current settings on machine 100. It also illustratively includes ahistoric display portion 440 that shows historic values (for the recenthistory) for the various settings. In addition, it includes a controlactuator 442 that can be actuated to navigate the user to another userinterface display with user input mechanisms that can be used to changethe settings displayed in current setting display portion 349. Itfurther includes a machine selector 444 and an “apply” actuator 446.Machine selector 444 can be actuated to select the different machines towhich the adjusted settings are to be applied, and the “apply” actuator446 is actuated to generate and send the control input to the selectedmachines, so that the adjusted settings can be applied to thosemachines.

FIG. 8 is similar to FIG. 7, except that the remote user 220 has nowactuated selector 434 to change the video or image information that isdisplayed so that the video image that is now being live streamed (orotherwise transmitted in near real time) to remote user computing system204 is video from the forward facing camera 156 on combine 100. It canbe seen in FIG. 8 that the context information displayed has notchanged, although, in another example, the context information maychange when the video information is changed.

The present discussion has mentioned processors and servers. In oneexample, the processors and servers include computer processors withassociated memory and timing circuitry, not separately shown. They arefunctional parts of the systems or devices to which they belong and areactivated by, and facilitate the functionality of the other componentsor items in those systems.

It will be noted that the above discussion has described a variety ofdifferent systems, components and/or logic. It will be appreciated thatsuch systems, components and/or logic can be comprised of hardware items(such as processors and associated memory, or other processingcomponents, some of which are described below) that perform thefunctions associated with those systems, components and/or logic. Inaddition, the systems, components and/or logic can be comprised ofsoftware that is loaded into a memory and is subsequently executed by aprocessor or server, or other computing component, as described below.The systems, components and/or logic can also be comprised of differentcombinations of hardware, software, firmware, etc., some examples ofwhich are described below. These are only some examples of differentstructures that can be used to form the systems, components and/or logicdescribed above. Other structures can be used as well.

Also, a number of user interface displays have been discussed. They cantake a wide variety of different forms and can have a wide variety ofdifferent user actuatable input mechanisms disposed thereon. Forinstance, the user actuatable input mechanisms can be text boxes, checkboxes, icons, links, drop-down menus, search boxes, etc. They can alsobe actuated in a wide variety of different ways. For instance, they canbe actuated using a point and click device (such as a track ball ormouse). They can be actuated using hardware buttons, switches, ajoystick or keyboard, thumb switches or thumb pads, etc. They can alsobe actuated using a virtual keyboard or other virtual actuators. Inaddition, where the screen on which they are displayed is a touchsensitive screen, they can be actuated using touch gestures. Also, wherethe device that displays them has speech recognition components, theycan be actuated using speech commands.

A number of data stores have also been discussed. It will be noted theycan each be broken into multiple data stores. All can be local to thesystems accessing them, all can be remote, or some can be local whileothers are remote. All of these configurations are contemplated herein.

Also, the figures show a number of blocks with functionality ascribed toeach block. It will be noted that fewer blocks can be used so thefunctionality is performed by fewer components. Also, more blocks can beused with the functionality distributed among more components.

FIG. 9 is a block diagram of the architecture, shown in FIG. 2, exceptthat harvester 100 communicates with elements in a remote serverarchitecture 500. In an example, remote server architecture 500 canprovide computation, software, data access, and storage services that donot require end-user knowledge of the physical location or configurationof the system that delivers the services. In various examples, remoteservers can deliver the services over a wide area network, such as theinternet, using appropriate protocols. For instance, remote servers candeliver applications over a wide area network and they can be accessedthrough a web browser or any other computing component. Software orcomponents shown in FIGS. 1 and 2 as well as the corresponding data, canbe stored on servers at a remote location. The computing resources in aremote server environment can be consolidated at a remote data centerlocation or they can be dispersed. Remote server infrastructures candeliver services through shared data centers, even though they appear asa single point of access for the user. Thus, the components andfunctions described herein can be provided from a remote server at aremote location using a remote server architecture. Alternatively, theycan be provided from a conventional server, or they can be installed onclient devices directly, or in other ways.

In the example shown in FIG. 9, some items are similar to those shown inFIGS. 1 and 2 and they are similarly numbered. FIG. 9 specifically showsthat remote server computing system 202 can be located at a remoteserver location 502. Therefore, harvester 100 accesses those systemsthrough remote server location 502.

FIG. 9 also depicts another example of a remote server architecture.FIG. 9 shows that it is also contemplated that some elements of FIG. 2are disposed at remote server location 502 while others are not. By wayof example, data store 262 or authentication system 264 can be disposedat a location separate from location 502, and accessed through theremote server at location 502. Regardless of where they are located,they can be accessed directly by harvester 100, through a network(either a wide area network or a local area network), they can be hostedat a remote site by a service, or they can be provided as a service, oraccessed by a connection service that resides in a remote location.Also, the data can be stored in substantially any location andintermittently accessed by, or forwarded to, interested parties. Forinstance, physical carriers can be used instead of, or in addition to,electromagnetic wave carriers. In such an embodiment, where cellcoverage is poor or nonexistent, another mobile machine (such as a fueltruck) can have an automated information collection system. As theharvester comes close to the fuel truck for fueling, the systemautomatically collects the information from the harvester using any typeof ad-hoc wireless connection. The collected information can then beforwarded to the main network as the fuel truck reaches a location wherethere is cellular coverage (or other wireless coverage). For instance,the fuel truck may enter a covered location when traveling to fuel othermachines or when at a main fuel storage location. All of thesearchitectures are contemplated herein. Further, the information can bestored on the harvester until the harvester enters a covered location.The harvester, itself, can then send the information to the mainnetwork.

It will also be noted that the elements of FIG. 2, or portions of them,can be disposed on a wide variety of different devices. Some of thosedevices include servers, desktop computers, laptop computers, tabletcomputers, or other mobile devices, such as palm top computers, cellphones, smart phones, multimedia players, personal digital assistants,etc.

FIG. 10 is a simplified block diagram of one illustrative example of ahandheld or mobile computing device that can be used as a user's orclient's hand held device 16, in which the present system (or parts ofit) can be deployed. For instance, a mobile device can be deployed inthe operator compartment of harvester 100 or that can be used as remoteuser computing system 204. FIGS. 11-12 are examples of handheld ormobile devices.

FIG. 10 provides a general block diagram of the components of a clientdevice 16 that can run some components shown in FIG. 2, that interactswith them, or both. In the device 16, a communications link 13 isprovided that allows the handheld device to communicate with othercomputing devices and under some embodiments provides a channel forreceiving information automatically, such as by scanning. Examples ofcommunications link 13 include allowing communication though one or morecommunication protocols, such as wireless services used to providecellular access to a network, as well as protocols that provide localwireless connections to networks.

In other examples, applications can be received on a removable SecureDigital (SD) card that is connected to an interface 15. Interface 15 andcommunication links 13 communicate with a processor 17 (which can alsoembody processors from other FIGS.) along a bus 19 that is alsoconnected to memory 21 and input/output (I/O) components 23, as well asclock 25 and location system 27.

I/O components 23, in one examples, are provided to facilitate input andoutput operations. I/O components 23 for various embodiments of thedevice 16 can include input components such as buttons, touch sensors,optical sensors, microphones, touch screens, proximity sensors,accelerometers, orientation sensors and output components such as adisplay device, a speaker, and or a printer port. Other I/O components23 can be used as well.

Clock 25 illustratively comprises a real time clock component thatoutputs a time and date. It can also, illustratively, provide timingfunctions for processor 17.

Location system 27 illustratively includes a component that outputs acurrent geographical location of device 16. This can include, forinstance, a global positioning system (GPS) receiver, a LORAN system, adead reckoning system, a cellular triangulation system, or otherpositioning system. It can also include, for example, mapping softwareor navigation software that generates desired maps, navigation routesand other geographic functions.

Memory 21 stores operating system 29, network settings 31, applications33, application configuration settings 35, data store 37, communicationdrivers 39, and communication configuration settings 41. Memory 21 caninclude all types of tangible volatile and non-volatilecomputer-readable memory devices. It can also include computer storagemedia (described below). Memory 21 stores computer readable instructionsthat, when executed by processor 17, cause the processor to performcomputer-implemented steps or functions according to the instructions.Processor 17 can be activated by other components to facilitate theirfunctionality as well.

FIG. 11 shows one example in which device 16 is a tablet computer 600.In FIG. 11, computer 600 is shown with user interface display screen602. Screen 602 can be a touch screen or a pen-enabled interface thatreceives inputs from a pen or stylus. It can also use an on-screenvirtual keyboard. Of course, it might also be attached to a keyboard orother user input device through a suitable attachment mechanism, such asa wireless link or USB port, for instance. Computer 600 can alsoillustratively receive voice inputs as well.

FIG. 12 shows that the device can be a smart phone 71. Smart phone 71has a touch sensitive display 73 that displays icons or tiles or otheruser input mechanisms 75. Mechanisms 75 can be used by a user to runapplications, make calls, perform data transfer operations, etc. Ingeneral, smart phone 71 is built on a mobile operating system and offersmore advanced computing capability and connectivity than a featurephone.

Note that other forms of the devices 16 are possible.

FIG. 13 is one example of a computing environment in which elements ofFIG. 2, or parts of it, (for example) can be deployed. With reference toFIG. 13, an example system for implementing some embodiments includes ageneral-purpose computing device in the form of a computer 810.Components of computer 810 may include, but are not limited to, aprocessing unit 820 (which can comprise processor 108), a system memory830, and a system bus 821 that couples various system componentsincluding the system memory to the processing unit 820. The system bus821 may be any of several types of bus structures including a memory busor memory controller, a peripheral bus, and a local bus using any of avariety of bus architectures. Memory and programs described with respectto FIG. 2 can be deployed in corresponding portions of FIG. 13.

Computer 810 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 810 and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media is different from, anddoes not include, a modulated data signal or carrier wave. It includeshardware storage media including both volatile and nonvolatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by computer 810. Communication media may embody computerreadable instructions, data structures, program modules or other data ina transport mechanism and includes any information delivery media. Theterm “modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal.

The system memory 830 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 831and random access memory (RAM) 832. A basic input/output system 833(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 810, such as during start-up, istypically stored in ROM 831. RAM 832 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 820. By way of example, and notlimitation, FIG. 13 illustrates operating system 834, applicationprograms 835, other program modules 836, and program data 837.

The computer 810 may also include other removable/non-removablevolatile/nonvolatile computer storage media. By way of example only,FIG. 13 illustrates a hard disk drive 841 that reads from or writes tonon-removable, nonvolatile magnetic media, an optical disk drive 855,and nonvolatile optical disk 856. The hard disk drive 841 is typicallyconnected to the system bus 821 through a non-removable memory interfacesuch as interface 840, and optical disk drive 855 are typicallyconnected to the system bus 821 by a removable memory interface, such asinterface 850.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Application-specific Integrated Circuits (e.g., ASICs),Application-specific Standard Products (e.g., ASSPs), System-on-a-chipsystems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 13, provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 810. In FIG. 13, for example, hard disk drive 841 isillustrated as storing operating system 844, application programs 845,other program modules 846, and program data 847. Note that thesecomponents can either be the same as or different from operating system834, application programs 835, other program modules 836, and programdata 837.

A user may enter commands and information into the computer 810 throughinput devices such as a keyboard 862, a microphone 863, and a pointingdevice 861, such as a mouse, trackball or touch pad. Other input devices(not shown) may include a joystick, game pad, satellite dish, scanner,or the like. These and other input devices are often connected to theprocessing unit 820 through a user input interface 860 that is coupledto the system bus, but may be connected by other interface and busstructures. A visual display 891 or other type of display device is alsoconnected to the system bus 821 via an interface, such as a videointerface 890. In addition to the monitor, computers may also includeother peripheral output devices such as speakers 897 and printer 896,which may be connected through an output peripheral interface 895.

The computer 810 is operated in a networked environment using logicalconnections (such as a local area network—LAN, or wide area network WAN)to one or more remote computers, such as a remote computer 880. Whenused in a LAN networking environment, the computer 810 is connected tothe LAN 871 through a network interface or adapter 870. When used in aWAN networking environment, the computer 810 typically includes a modem872 or other means for establishing communications over the WAN 873,such as the Internet. In a networked environment, program modules may bestored in a remote memory storage device. FIG. 13 illustrates, forexample, that remote application programs 885 can reside on remotecomputer 880. It should also be noted that the different examplesdescribed herein can be combined in different ways. That is, parts ofone or more examples can be combined with parts of one or more otherexamples. All of this is contemplated herein.

Example 1 is an agricultural harvesting machine, comprising:

-   -   a visual information capture mechanism that captures visual        information;    -   context information correlation logic that identifies context        information corresponding to the visual information;    -   output generator logic configured to generate an output        including the context information and corresponding visual        information; and    -   a transmission system that transmits the output to a remote        computing system, remote from the agricultural harvesting        machine, for display of the context information and the        corresponding visual information.

Example 2 is the agricultural harvesting machine of any or all previousexamples wherein the visual information capture mechanism comprises:

-   -   a plurality of video cameras mounted on the agricultural        harvesting machine and each capturing video information.

Example 3 is the agricultural harvesting machine of any or all previousexamples wherein first context information corresponds to the videoinformation from a first of the plurality of video cameras and secondcontext information corresponds to the video information from a secondof the plurality of video cameras, the first context information beingdifferent from the second context information.

Example 4 is the agricultural harvesting machine of any or all previousexamples and further comprising:

-   -   a plurality of sensors each generating a sensor signal        indicative of a sensed variable; and    -   a control system generating the context information based on the        sensor signals.

Example 5 is the agricultural harvesting machine of any or all previousexamples and further comprising:

-   -   trigger detection logic configured to detect a trigger        indicative of when the output including the context information        and corresponding video information is to be transmitted to the        remote computing system and generating a trigger signal        identifying the video information based on detection of the        trigger.

Example 6 is the agricultural harvesting machine of any or all previousexamples wherein the trigger detection logic comprises:

-   -   a request detector detecting a request from the remote computing        system, the request identifying which video information is to be        transmitted.

Example 7 is the agricultural harvesting machine of any or all previousexamples wherein the correlation logic is configured to identify thecorresponding context information, that corresponds to the identifiedvideo information, for transmission with the identified videoinformation.

Example 8 is the agricultural harvesting machine of any or all previousexamples wherein the trigger detection logic comprises:

-   -   a continuous transmission detector generating the trigger signal        indicating that the video information and corresponding context        information is to be sent continuously, the transmission system        being configured to continuously transmit the identified video        information and corresponding context information to the remote        computing system based on the trigger signal.

Example 9 is the agricultural harvesting machine of any or all previousexamples wherein the trigger detection logic comprises:

-   -   an event detector detecting an event and generating the trigger        signal indicating that the video information and corresponding        context information is to be sent to the remote computing system        based on the detected event, the transmission system being        configured to continuously transmit the identified video        information and corresponding context information to the remote        computing system based on the trigger signal.

Example 10 is a method of controlling an agricultural harvestingmachine, comprising:

-   -   capturing video information using a video capture mechanism on        the agricultural harvesting machine;    -   generating a sensor signal indicative of a sensed variable;    -   identifying context information, based on the sensor signal,        corresponding to the video information;    -   generating an output including the context information and        corresponding video information; and    -   transmitting the output to a remote computing system, remote        from the agricultural harvesting machine, for display of the        context information and the corresponding video information.

Example 11 is the method of any or all previous examples whereincapturing video information comprises:

-   -   capturing video information from each of a plurality of        different video cameras mounted on the agricultural harvesting        machine.

Example 12 is the method of any or all previous examples whereinidentifying context information comprises:

-   -   identifying first context information corresponding to the video        information from a first of the plurality of different video        cameras; and    -   identifying second context information corresponding to the        video information from a second of the plurality of video        capture mechanisms, the first context information being        different from the second context information.

Example 13 is the method of any or all previous examples whereingenerating a sensor signal comprises:

-   -   generating a plurality of different sensor signals, each        indicative of a different sensed variable; and    -   generating the context information based on the plurality of        different sensor signals.

Example 14 is the method of any or all previous examples and furthercomprising:

-   -   detecting a request from the remote computing system; and    -   identifying, based on the request, which video information is to        be transmitted, wherein identifying context information includes        identifying the corresponding context information, that        corresponds to the identified video information, for        transmission with the identified video information.

Example 15 is the method of any or all previous examples whereintransmitting the output to the remote computing system comprises:

-   -   continuously transmitting the identified video information and        corresponding context information to the remote computing        system.

Example 16 is the method of any or all previous examples and furthercomprising:

-   -   detecting an event; and    -   identifying which video information and corresponding context        information is to be sent to the remote computing system based        on the detected event, wherein transmitting comprises        continuously transmitting the identified video information and        corresponding context information to the remote computing system        based on the detected event.

Example 17 is a mobile device, comprising:

-   -   machine selector logic that displays a user actuatable machine        selection element that is actuatable to select a remote        agricultural harvesting machine, that is remote from the mobile        device;    -   a communication system that communicates with a selected remote        agricultural harvesting machine;    -   video information display logic that displays near real time        video information received from a video camera on the selected        remote agricultural harvesting machine; and    -   context information display logic that displays near real time        context information, corresponding to the video information,        received from the selected agricultural harvesting machine and        generated based on sensor signals generated by sensors on the        selected agricultural harvesting machine.

Example 18 is the mobile device of any or all previous examples whereinthe machine selector logic is configured to display a plurality ofdifferent machine selector elements that are each actuatable to select adifferent remote agricultural harvesting machine, and furthercomprising:

-   -   a user interaction detector configured to detect user actuation        of a plurality of the different machine selector elements to        select a plurality of different remote agricultural harvesting        machines.

Example 19 is the mobile device of any or all previous examples whereinthe video information display logic is configured to display near realtime video information received from a video camera on each of theplurality of selected remote agricultural harvesting machines andwherein the context information display logic is configured to displaynear real time context information, corresponding to the videoinformation, received from each of the selected agricultural harvestingmachines and generated based on sensor signals generated by sensors oneach of the selected remote agricultural harvesting machines.

Example 20 is the mobile device of any or all previous examples andfurther comprising:

-   -   remote control generator logic configured to generate a user        actuatable remote control element that is actuated to send a        control signal to remotely control the selected remote        agricultural harvesting machine.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. An agricultural harvesting machine, comprising: acrop processing system configured to process a crop that is gathered bythe agricultural harvesting machine; a visual information capturemechanism that captures visual information; context informationcorrelation logic that identifies context information corresponding tothe visual information; output generator logic configured to generate anoutput including the context information and corresponding visualinformation; and a transmission system that transmits the output to aremote computing system, remote from the agricultural harvestingmachine, for display of the context information and the correspondingvisual information.
 2. The agricultural harvesting machine of claim 1wherein the visual information capture mechanism comprises: a pluralityof video cameras mounted on the agricultural harvesting machine and eachcapturing video information.
 3. The agricultural harvesting machine ofclaim 2 wherein first context information corresponds to the videoinformation from a first of the plurality of video cameras and secondcontext information corresponds to the video information from a secondof the plurality of video cameras, the first context information beingdifferent from the second context information.
 4. The agriculturalharvesting machine of claim 3 and further comprising: a plurality ofsensors each generating a sensor signal indicative of a sensed variable;and a control system generating the context information based on thesensor signals.
 5. The agricultural harvesting machine of claim 4 and,further comprising: trigger detection logic configured to detect atrigger indicative of when the output including the context informationand corresponding video information is to be transmitted to the remotecomputing system and generating a trigger signal identifying the videoinformation based on detection of the trigger.
 6. The agriculturalharvesting machine of claim 5 wherein the trigger detection logiccomprises: a request detector detecting a request from the remotecomputing system, the request identifying which video information is tobe transmitted.
 7. The agricultural harvesting machine of claim 1,wherein the captured video information represents the crop in a cropprocessing system of the agricultural harvesting machine, and whereinthe correlation logic is configured to identify the correspondingcontext information, that corresponds to the identified videoinformation, for transmission with the identified video information. 8.The agricultural harvesting machine of claim 5 wherein the triggerdetection logic comprises: a continuous transmission detector generatingthe trigger signal indicating that the video information andcorresponding context information is to be sent continuously, thetransmission system being configured to continuously transmit theidentified video information and corresponding context information tothe remote computing system based on the trigger signal.
 9. Theagricultural harvesting machine of claim 5 wherein the trigger detectionlogic comprises: an event detector detecting an event and generating thetrigger signal indicating that the video information and correspondingcontext information is to be sent to the remote computing system basedon the detected event, the transmission system being configured tocontinuously transmit the identified video information and correspondingcontext information to the remote computing system based on the triggersignal.
 10. A method of controlling an agricultural harvesting machine,the method comprising: performing, in a crop processing system of theagricultural harvesting machine, a crop processing operation on a cropgathered by the agricultural harvesting machine; capturing videoinformation using a video capture mechanism on the agriculturalharvesting machine, the captured video information representing the cropin the crop processing system; generating a sensor signal indicative ofa sensed variable; identifying context information, based on the sensorsignal, corresponding to the video information; generating an outputincluding the context information and corresponding video information;and transmitting the output to a remote computing system, remote fromthe agricultural harvesting machine, for display of the contextinformation and the corresponding video information.
 11. The method ofclaim 10 wherein capturing video information comprises: capturing videoinformation from each of a plurality of different video cameras mountedon the agricultural harvesting machine.
 12. The method of claim 11wherein identifying context information comprises: identifying firstcontext information corresponding to the video information from a firstof the plurality of different video cameras; and identifying secondcontext information corresponding to the video information from a secondof the plurality of video capture mechanisms, the first contextinformation being different from the second context information.
 13. Themethod, of claim 12 wherein generating a sensor signal comprises:generating a plurality of different, sensor signals, each indicative ofa different sensed variable; and generating the context informationbased on the plurality of different sensor signals.
 14. The method ofclaim 13 and further comprising: detecting a request from the remotecomputing system; and identifying, based on the request which videoinformation is to be transmitted, wherein identifying contextinformation includes identifying the corresponding context information,that corresponds to the identified video information for transmissionwith the identified video information.
 15. The method of claim 14wherein transmitting the output to the remote computing systemcomprises: continuously transmitting the identified video informationand corresponding context information to the remote computing system.16. The method of claim 13 and further comprising: detecting an event;and identifying which video information and corresponding contextinformation is to be sent to the remote computing system based on thedetected event, wherein transmitting comprises continuously transmittingthe identified video information and corresponding context informationto the remote computing system based on the detected event.
 17. A mobiledevice, comprising: machine selector logic that displays a useractuatable machine selection element that is actuatable to select aremote agricultural harvesting machine, that is remote from the mobiledevice; a communication system that communicates with a selected remoteagricultural harvesting machine; video information display logic thatdisplays near real time video information received from a video cameraon the selected remote agricultural harvesting machine; and contextinformation display logic that displays near real time contextinformation, corresponding to the video information, received from theselected agricultural harvesting machine and generated based on sensorsignals generated by sensors on the selected agricultural harvestingmachine.
 18. The mobile device of claim 17 wherein the machine selectorlogic is configured to display a plurality of different machine selectorelements that are each actuatable to select a different remoteagricultural harvesting machine, and further comprising: a userinteraction detector configured to detect user actuation of a pluralityof the different machine selector elements to select a plurality ofdifferent remote agricultural harvesting machines.
 19. The mobile deviceof claim 18 wherein the video information display logic is configured todisplay near real time video information received from a video camera oneach of the plurality of selected remote agricultural harvestingmachines and wherein the context information display logic is configuredto display near real time context information, corresponding to thevideo information, received from each of the selected agriculturalharvesting machines and generated based on sensor signals generated bysensors on each of the selected remote agricultural harvesting machines.20. The mobile device of claim 17 and further comprising: remote controlgenerator logic configured to generate a user actuatable remote controlelement that is actuated to send a control signal to remotely controlthe selected remote agricultural harvesting machine.